System and method for multi-directional positioning of projected images

ABSTRACT

A system and method of multi-directional projection in which the optical output from a projector is modified in response to characteristics of nearby surfaces, upon which projection can be directed. The multi-directional projection system can provide beneficial output as a primary indicator or a secondary indicating device that can be controlled by itself or synchronously with other devices, such as output from a computer monitor. One implementation incorporates an optical sensor coupled to the projector for registering object characteristics (e.g., distance, reflectance, texture, color, and ambient lighting), such as during a mapping operation performed across the pan and tilt range of the projector. The projection system preferably incorporates a focusing assembly and may include other controllable optical elements such as zoom. Control is preferably provided through commands within an application programming interface (API), such as executing on a personal computer or other form of computer device.

CROSS-REFERENCE TO RELATED APPLICATIONS

Not Applicable

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not Applicable

INCORPORATION-BY-REFERENCE OF MATERIAL SUBMITTED ON A COMPACT DISC

Not Applicable

NOTICE OF MATERIAL SUBJECT TO COPYRIGHT PROTECTION

A portion of the material in this patent document is subject tocopyright protection under the copyright laws of the United States andof other countries. The owner of the copyright rights has no objectionto the facsimile reproduction by anyone of the patent document or thepatent disclosure, as it appears in the United States Patent andTrademark Office publicly available file or records, but otherwisereserves all copyright rights whatsoever. The copyright owner does nothereby waive any of its rights to have this patent document maintainedin secrecy, including without limitation its rights pursuant to 37C.F.R. §1.14.

BACKGROUND OF INVENTION

1. Field of Invention

This invention pertains generally to image projection, and moreparticularly to a system and method for controlling the location towhich an image is projected and modifying the projected image based oncharacteristics of that location.

2. Description of Related Art

Conventional projectors are configured with a fixed projection elementfor projecting images, ranging from still images to full-motion videoimages, in a fixed direction. Typically these projectors require manualadjustment to properly align the output to a rectangular viewing screen.The output of these projectors is configured for being directed at aflat surface that is suitably sized and of a suitable material toproperly reflect the light back to the viewer to provide a clear,non-distorted view.

These conventional projectors are typically utilized in the same mannerand for the same purposes as computer monitors or television displayscreens. However, in developing the system and method taught herein, ithas been recognized that conventional projectors are unable to providean active viewing experience that surrounds the viewer.

Therefore, a need exists for a projection system and method that allowsimages to be projected at different locations in the surroundings of theviewer. The present invention satisfies those needs, as well as others,and overcomes the deficiencies of previously developed fixed-positionprojection systems.

BRIEF SUMMARY OF INVENTION

A projection system and method is described which provides forprojection of images, ranging from still images to full motion videoimages, within an environment that can contain irregular surfaces (e.g.,irregular with regard to distance, angle, surface material, color,lighting, size, shape, and so forth) onto which an image is projected.More particularly, the system and method provides for multi-positionalcontrol of the location to which images are projected as well as forimage modification based on characteristics of the location. By way ofexample, and not of limitation, an aspect of the system and method isthat the direction of image projection can be changed under programcontrol in one or more dimensions, such as by changing horizontal andvertical projection angles (e.g., pan and tilt). Another aspect of thesystem and method is that the image is modified based on characteristicsof the location to which the image is projected. Another aspect of thesystem and method is to access those characteristics using a map whichhas been loaded into a memory storage area accessible to the system. Themap maintains characteristics of surrounding surfaces obtained from amapping operation performed, for example, during a setup process. Duringoperation, software retrieves characteristic values from within the mapfor the projection direction (e.g., indexed by pan and tilt anglevalues) and modifies the projected image to compensate forcharacteristics of the surroundings onto which the image is beingprojected.

The term “projection system” is generally used herein to describe acombination of a projection head and a projection control module. In oneembodiment, the projection head comprises a projection element forprojecting an image, as well as one or more motion control elements(e.g., actuators) for changing the direction of image output from theprojection element. The projection control module comprises a controlprocessor, such as a computer, that communicates with the projectionhead and controls its motion to project an image, either smoothly orstepwise, onto different projection locations (e.g., first location,second location, third location, . . . nth location) based oninformation retrieved from a map of projection location characteristics.Motion can be similarly directed in response to giving a direction and atime or distance value, instead of the discrete location valuesdescribed above. Input for controlling the projection system is receivedby the projection control module such as from application programming.

One aspect of the projection system and method is the capability todisplay images which are not constrained to being projected onto arectangular display screen or area. This projection system and methodallows for the display of text, graphics, images (ranging from stillimages to full-motion video), and combinations thereof either staticallyor in motion on surfaces in the vicinity of projection.

In one embodiment, an image projection element provides the directedlight output for the projection system. The image projection element isconfigured for directing the light output across a range of directions,using, for example, one or more motion control elements such asmotorized actuators to rotate the image projection element along one ormore axes, such as horizontally (pan) and vertically (tilt). Dependingon the specific application, the projected images can be used as aprimary (main) output from the application or as a secondary (auxiliary)output; for example, to operate in a synchronized manner with a primaryoutput such as a conventional flat panel display screen. When operatingas a secondary display, the projection techniques described hereinoperate without interfering with output from the primary output, andprovide additional display output to be seen by the user withinlocations surrounding the projection head. According to the teachingsherein, any number of projection heads can be controlled by a projectioncontrol module depending on the configuration of the applicationprogram.

A number of advantages are provided by the system and method describedherein. In contrast to the image output provided by a fixed paneldisplay, this projection system allows image output to be readilydisplayed anywhere, providing information where it is needed as adynamic display or accompaniment to conventional display screens,control of optically sensitive devices, and other uses. In oneembodiment, this projection system is configured to provide one or moreadditional application displays without “interrupting” the main userdisplay screen.

The projection system is suitable for use in any application thatrequires display of an image (e.g., still, sequence, and/or video). Suchan image can be associated with, for example, passive entertainment,interactive entertainment, commercial information technology (IT),personal computer (PC) applications, and communications. The followingare provided by way of example and not by way of limitation.Entertainment uses include: “move to the music (or video)”, video,gaming, game play information, text, video chat, game integration (e.g.,laser shots, game stats, sprites, creatures, and so forth). Examples inthe IT area include: instant message (IM) text, phone number display,image display of persons, general info, weather, stock prices, and soforth. Examples with email include: email arrival notification, emailtext (static/scrolling) information that is “pushed” to users over anetwork, and other applications where directed output is beneficial.

The projection system described herein can also be used in a widevariety of situations and locations. By way of example, and notlimitation, these locations include: family room (entertainment center),home PC, office PC, mobile, cell phone, laptop as well as other desiredlocations.

The teachings herein are amenable to embodiment in a number of ways,including but not limited to the following descriptions.

In one embodiment, an image projection system according to the teachingsherein includes (i) an image projection element, (ii) motion controlelement, coupled to the image projection element and configured formoving an image projected by the image projection element from a firstprojection location to a second projection location, and (iii) a controlprocessor configured for receiving an electrical source signal andconverting the signal to an image for projection by the image projectionelement, actuating the motion control element in response to receipt ofa motion control command, and modifying the electrical source signalaccording to a map of projection location characteristics.

In one embodiment, the image projection element includes a light source,a modulation element configured for modifying intensity, color, or bothintensity and color, of light from the light source, and an opticalassembly configured for projecting the light from the image projectionelement, wherein the light source and the modulation element convert theelectronic source signal into the image for projection by the opticalassembly.

In one mode, the motion control element controls horizontal angularmotion. In another mode, the motion control element controls verticalangular motion. In another mode, the motion control element controlsboth horizontal angular motion and vertical angular motion.

In another embodiment, an image projection method according to theteachings herein includes (i) projecting an image onto a firstprojection location, (ii) receiving a command to project the image ontoa second projection location, (iii) moving the image from the firstprojection location to the second projection location, and (iv)modifying the image in accordance with mapped characteristics associatedwith the second projection location.

In one mode, modifying the image comprises changing opticalcharacteristics of projection. In another mode, modifying the imagecomprises modifying signals carrying the image prior to projecting theimage onto the second projection location. In another mode, modifyingthe image comprises both changing the optical characteristics ofprojection and modifying the signals carrying the image.

In one implementation, a projection system according to the teachingsherein includes the following: (a) an image projection elementconfigured for projecting an image; (b) a motion control element, orelements, coupled to the image projection element, for changing thedirection of the projected image from a first projection location to asecond projection location; (c) a computer processor; and (d)programming executable on the computer processor for carrying out theoperations of (i) generating a map of characteristics for projectionsurfaces within the surroundings of the image projection element, (ii)receiving an electrical source signal for conversion to an opticaloutput on the image projection element, (iii) controlling the motioncontrol element(s) to change the direction of the optical output inresponse to the receipt of a motion command, and (iv) modifying theelectrical source signal according to characteristics stored within themap for the direction of the optical output. In one embodiment, the mapis pre-generated and is accessed by the programming. In one embodiment,the image is stored in the map.

In another implementation, a projection method includes: (a) mappingareas surrounding an image projection element (e.g., within a projectionhead) to create a map of surfaces onto which an image can be projected;(b) conversion of an electrical signal to a projected optical outputfrom the image projection element; (c) changing the direction of theprojected optical output emanating from the image projection element inresponse to motion commands; and (d) modifying the projected opticaloutput by utilizing correction factors determined from informationobtained from the map for the direction of projected optical output.

In another implementation, a projection method includes: (a) projectingan image from an image projection element onto a first projectionlocation; (b) receiving a command to move the image onto a secondprojection location; (c) activating a motion control element coupled tothe projection element to move the output image from the firstprojection location to the second projection location; and (d) modifyingimage output in response to accessing mapped characteristics associatedwith the second projection location. The image output can be modified bychanging optical properties of the projection element, and/or byperforming image processing on the image signals prior to beingprojected.

Embodiments of the system and method can provide a number of beneficialaspects which can be implemented either separately or in any desiredcombination without departing from the present teachings.

For example, one aspect of the invention is to provide for changinglocations onto which images are projected from a projector which can becontrolled from an application program.

Another aspect is to provide for changing locations onto which imagesare projected by a projector in which the motion is synchronized with anapplication program which may be outputting additional information onother output devices, such as video display screens.

Another aspect is to provide for projection of images on surfaces nearbya projector which can be irregular in shape, slope, composition, andlocated at varying distances.

Another aspect is to provide for projection of images based on mappingof the surrounding projection areas so that correction can be applied tothe projected output in response to the map for any given outputdirection.

Another aspect is to provide for projection of images in which opticaloutput modification, such as focus and zooming operations, can becontrolled in response to commands from the application in combinationwith map information.

Another aspect is to provide a projection system in which an opticalsensor coupled to the device registers characteristics of thesurroundings.

Another aspect is to provide a projection system in which the opticalsensor is mounted to, or integrated with, the projector element, whereinthe projection apparatus and sensor move together pointing in theprojection direction.

Another aspect is to provide a projection system in which the opticalsensor provides distance information on objects in the surroundings ofthe projection apparatus.

Another aspect is to provide a projection system in which the opticalsensor can register numerous characteristics of the surroundings, suchas distance, reflectivity, texture, color, ambient lighting and otherdesired characteristics.

Another aspect is to provide a projection system in which the output ofthe projection apparatus can be directed onto a nearby electronicdevice, which is configured for optical remote control, forcommunicating remote control commands to that device.

Another aspect is to provide a projection system in which commandswithin an application programming interface (API) layer can be accessedby any application programs executing on the computer.

Another aspect is to provide a projection system to display text,graphics, images, icons, simple vector graphics, and to output opticalsequences for controlling devices configured for receiving opticalinputs.

Another aspect is to provide a projection system to augment the displaysfor gaming, video presentation, slide presentation, conference calls,phone calls, text messaging, instant messaging, e-mail messages,information displays (e.g., weather, stocks, sports, email arrivalindicator, and so forth), and other applications in which additionaloptical output is desired.

Another aspect is to provide a projection system in which the motion isperformed responsive to one or more audio and/or video streams beingoutput on another device, such as a computer display.

A still further aspect is to provide a projection system in which theprojection output and motions can be controlled remotely by a local useror a remote user such as over the internet.

Further aspects of the system and method will be brought out in thefollowing portions of the specification, wherein the detaileddescription is for the purpose of fully disclosing preferred embodimentsof the invention without placing limitations thereon.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

The invention will be more fully understood by reference to thefollowing drawings which are for illustrative purposes only:

FIG. 1 is a block diagram of a projection motion control systemaccording to an embodiment of the present teachings, showing theprojection apparatus and control programming elements.

FIG. 2A is a top schematic view of projection illustrating horizontalmotion (pan) according to an aspect of the present teachings.

FIG. 2B is a side schematic view of projection illustrating verticalmotion (tilt) according to an aspect of the present teachings.

FIG. 3 is a front view of a projection apparatus according to anembodiment of the present teachings.

FIG. 4 is a block diagram of a projection motion control systemembodiment of the present teachings, showing multi-directionalprojection output with integral optical sensor within the head of theprojection apparatus.

FIG. 5 is a flow diagram of general steps in the multi-directionalprojection method according to an aspect of the present teachings.

FIG. 6 is a flow diagram of a method for moving an image between a firstprojection location and a second projection location according to anaspect of the present teachings.

DETAILED DESCRIPTION OF INVENTION

Referring more specifically FIG. 1 through FIG. 6, an embodiment of theinvention will now be described in detail. It will be appreciated thatthe system may vary as to configuration and as to details of the parts,and that the method may vary as to the specific steps and sequence,without departing from the basic concepts as disclosed herein.

FIG. 1 illustrates an embodiment of a multi-directional outputprojection system 10 having a computer 12 coupled to a projection head14. Although shown as a wired connection, it should be appreciated thatcomputer 12 can alternatively communicate with projection head 14 over anetwork connection, wireless connection, or using power-linecommunication mechanisms.

Software for controlling the projection system is shown associated withcomputer 12, and includes a projector control module 16, setup module18, source application 20, and map based image processing routine 22.

Projection head 14 is shown as including a stationary base member 24upon which is attached a movable upper housing 26, such as rotatablycoupled to one another to provide a first degree of motion (e.g., pan).A motion control element, such as depicted as a first actuator (A1) 28,is configured to impart motion to upper housing 26, such as rotatingupper housing 26 in relation to stationary base member 24. It should benoted that stationary base member 24 can be mounted in any desiredrelationship to the movable upper housing 26 (e.g., above, below, to theside, and so forth), depending on the application and specificconditions.

A swivel assembly 30 within upper housing 26 is adapted for rotation,such as through axis 32, to impart a second degree of motion (e.g.,tilt) in response to the operation of a motion control element depictedas a second actuator (A2) 34. An image projection element 36 isconfigured for converting received image signals to an optical output.

A lens assembly 38 is shown coupled to image projection element 36 forfocusing the output of the projector. Additional control assemblies canbe coupled to image projection element 36 for controlling additionaldegrees of motion or other parameters. In the implementation shown, athird actuator (A3) 40 is shown for controlling the focus of lensassembly 38, and a fourth actuator (A4) 42 is shown for controlling thezoom of lens assembly 38. An asterisk ‘*’ is shown within elements 4046of the figure to clearly denote these elements are optional.

It should be appreciated that the motion control elements describedabove can be implemented with any desired form of electromechanicalactuator elements, such as utilizing motor drive assemblies (e.g.,geared motors), muscle wires, piezo-motors, and so forth for controllinghorizontal angular displacements (pan), vertical angular displacements(tilt), and optionally other controls such as focus and zoom.

It will be appreciated that the system can incorporate any desirednumber of optional features. For example, optical image modificationassemblies can be operably coupled to the projector element, such as afocusing lens, zoom lens, or other optical elements and combinationsthereof. Incorporating a focusing assembly, such as a focusing lens withfocusing actuator, provides a mechanism for changing the focus of theimage projector based on distance to the surface upon which the outputis projected. In a similar manner, a zoom lens with actuator can beincorporated and controlled for changing the size and intensity bychanging the angle subtended by the projection cone of the opticaloutput in response to programming based on the map and/or commandsreceived from application programming.

By way of example, and not limitation, a sensor 44 is shown forregistering characteristics of surfaces within the vicinity ofprojection head 14. The sensor can facilitate mapping of surfaces in thevicinity of the projection system. In one implementation sensor 44 is animage sensor which operates through the same lens as projector element36. Alternatively, sensor 44 can be implemented as a separate imagesensor having its own optics. As still another example, sensor 44 maycomprise a distance registration sensor (i.e., ultrasonic, laser). Itshould be appreciated that sensor 44 can comprise any sensor configuredfor detecting characteristics of projection surfaces in the vicinity ofprojection head 14.

Another optional feature is exemplified as a pixel redirection device,such as a mirror array 46, which allows the system to simultaneouslydirect any desired collections or groups of pixels within an image todifferent locations offset from the projection element direction. Thisoption allows the projector to create distributed and/or sparse effectsspanning a larger distance. Considering an example of use within a videogame, multiple objects can be shown moving about in the surroundingswithin a field of view that otherwise exceeds a conventional projectioncone. This option also allows multiple elements to be output within asingle image, for instance a plurality of icons or sprites (e.g., movingicons), that are then moved by the mirror array, while the projectiondirection can be stationary or traverse a path.

An optional user interface 48 is shown coupled to the projection head 14for providing status information and interaction with a user. The userinterface can include any desired input selectors, (e.g., buttons,switches and the like), and/or output indicators (e.g., powerindicators, status LEDs, displays, audio annunciators, and so forth).

The following describes an embodiment of the software of the system ingreater detail. Still referring to FIG. 1, setup module 18 is shown forconfiguring the projector, initializing it, inputting a map of thesurroundings and other related functions. In one mode, the map of thesurroundings can be input to the projection system in the form of a “MapDescription File” which details the surfaces with respect to thelocation of the projection element. Alternatively, or additionally, amap can be generated by the system by traversing a set of directions andprocessing information collected by sensor 44, such as performed duringthe setup process.

The software processing steps include the loading of, or alternativelythe creating of, a map of characteristics for nearby surfaces within thememory for access by the computer. Preferably this map descriptionincludes information within at least the movable range of projection.One embodiment of the map is configured to represent projectionlocations as respective pan and tilt angles. Another embodiment of themap is configured to represent room size (e.g., assume rectangular box),projector position within the room, and room objects (position, shape).It should be recognized, however, that any convenient representationapparatus can be utilized.

Considering the mapping of projection surfaces based on pan and tiltangles, it will be appreciated that the map may be implemented in a twodimensional array for retaining characteristic data for the differentpan and tilt values (at whatever angular resolution desired).Furthermore, to reduce processing overhead, a pointer can be used foraccessing fields within a memory block retained for the map. In this waythe map is accessed using retrieval based on a pointer generated inresponse to pan and tilt values. For example, left-shifting the panvalue by x bits (i.e., multiplication), then left-shifting the tiltvalue by y bits (where y<x) then adding a field offset value, whereinthe desired projection parameter can be retrieved via pointer for therespective projection direction.

One or more source application programs 20 are shown in FIG. 1 which areadapted for executing on computer 12, or alternatively another computerin communication with computer 12, and generating image data for outputby the projector as well as generating control commands for controllingthe movement and operations of the projector. In the example shown, anelectrical source signal containing the image, image stream, or video iscommunicated from application program 20 to projector control module 16.In addition, control commands are passed to the projector forcontrolling motion, and image effects. The output of the projectionsystem can be controlled by local programming, such as applicationprogramming executed on an associated personal computer, or control canbe performed from a remote device, such as over a dedicated controlline, a private intranet, or over the public internet, and the like.

By way of example, and not limitation, a number of application programtypes are particularly well suited for primary or auxiliary use of thedescribed multi-directional projection, such as: video games, videopresentation, slide presentation, conference calls, phone calls, textmessaging, instant messaging, e-mail messages, information displays(e.g., weather, stocks, sports, email arrival indicator, and so forth),and other applications that can benefit from creating externalmulti-directional optical projections. It should be appreciated that theapplications can be configured to operate the projection system as asole display output, or as an output to augment other forms of output,such as generated by a video monitor, audio system and so forth.

Map/image processing routine 22 within projector control module 16 isconfigured for changing the projected output in response to mapcharacteristics for a given output direction. In one mode, the imageprocessing routine processes the output based on the projected directionwhile taking into account the respective size of the projected image,because the projected output may extend across surfaces having differentcharacteristics and at different distances from the projector. In onemode, map/image processing routine 22 modulates the focus of the imagein response to distance information within the map. In another mode, theimage processor modifies the image signal and also controls the focusbased on correction factors determined for the current mode of operationfrom information retrieved from the map.

In one mode, the image processing software controls and/or modifies theimages or video signals so that images are projected correctly to suitthe different surfaces within the space surrounding the projection head.Modification of projected output is performed based on a map ofcharacteristics for the area surrounding the projector element. The goalis to correctly project the image that is given to the projector module.In one preferred mode of operation no image processing is performedunless required to modify the image to display correctly in the currentposition within the mapped space. The image is only changed to projectonto a surface that changes as the projection direction is changed.

During operation, the projector programming controls the motion controlelement(s) to change the direction of the optical output in response tothe receipt of a motion command and it modifies the optical output fromthe projector, the electrical source signal, or both optical and signalmodification, based on the characteristics found in the map for thegiven projection direction.

It should be appreciated that elements (e.g., actuators) withinprojector unit 14 can be directly controlled through external signals,as shown, or received by control circuitry of an interface forcommunicating between the projector unit and computer 12.

FIGS. 2A through 2B illustrate an example of pan and tilt ranges, suchas covering a full range of pan and tilt (360° by 360°) or any desiredportion thereof such as 360° pan and 270° tilt as shown in the figures,through which the projected output can be moved in response to controlof the motion control elements. It should be appreciated that theactuator mechanisms can be configured to produce any desired range ofmotion for a given application.

FIG. 3 illustrates an example physical embodiment of a projection head14. Stationary base member 24 is rotatably coupled to a movable upperhousing 26 which is configured for rotation through a horizontal angulardisplacement (pan). A swiveling portion 30 of upper housing 28incorporates the projection element having lens 38 as seen in thefigure. In response to swiveling of swiveling portion 30, the projectionelement can be moved through vertical angular displacements (tilt).Projection head 14 is configured for receiving both image and controlsignals, for example a first input for receiving an image or videostream and a second input for receiving motion and focus commands.Furthermore, additional commands and status can be communicated betweenthe projection unit and computer depending on the specific hardware andapplications being supported.

An optional user interface 48 is shown by way of example integrated intobase 24, although it can be alternatively integrated within any portionof the system, or coupled to the system, or based on a wireless remotecontrol and any desired combinations thereof. The user interfacefacilitates displaying status to the user and/or collecting user inputfor the projection system. By way of example and not limitation, userinterface 48 contains input selectors (e.g., switches, buttons) and/oroutput indicators (e.g., LED power light, status light, segmenteddisplay, audio annunciators, and so forth).

FIG. 4 illustrates an example embodiment 50 for multi-directionalprojection with an integral sensor for detecting characteristics ofobjects within the vicinity of the projector. Computer 52 is shown ashaving control programming 54 for controlling projection head 56including projection element 58 which can be moved through a range ofdirections in response to activating actuators 60 to change projectedoutput direction. In addition, at least one sensor 62 is incorporatedfor sensing characteristics of the surroundings which could impactprojection quality. Setup module 18 is shown coupled to map builderprogramming 64, while image processing module 22 is shown coupled tomemory 66 shown for storing the map and operation sequences. Acommunication interface 68 is shown for communicating between projectionhead 56 and computer 52, for example communicating status and datadigitally for controlling the operation of actuators and returningstatus information. It should be appreciated that the control commandsand image data can be communicated using any convenient communicationprotocol and connection type, such as using either a wired or wirelessconnection. Control and image data are shown communicated separatelybetween computer 52 and projection head 56, although communication canbe alternatively implemented combining these signals into a singledigital stream.

Control programming 54, such as implemented as one or more applicationprogram interfaces (APIs), is shown receiving image and control datafrom different application programs 20 a, 20 b. Control operationsexecuted by control programming 54 can be optionally synchronized basedon a timing signal, represented by receipt of a time signal which canprovide real time, time based on GPS signals, or other forms of timeevents to which operations are to be synchronized.

Sensor 62 is configured for registering characteristics of nearbysurfaces upon which the image projection element can be directed. Anyconvenient sensor can be utilized for registering these characteristics,such as separate photo detectors to detect reflected light intensity, anoptical imager (e.g., camera element) to detect any of a number ofsurface characteristics, a rangefinding device, or other devices capableof providing information about the surroundings. The sensor can bemounted with its own motion control actuators, or can utilize the motioncontrol actuators of the projector element. Mounting the sensor to theprojector element, or using a sensor integrated with the projectorelement, assures that the projector and sensor move together, thusreducing the need for separate motion control devices for the sensor andeliminating the need to translate sensor directions into projectiondirections.

It should be appreciated that using a form of optical imager as thesensor provides a number of benefits, such as that it can be selectivelyfocused to determine distance, while image information obtained from itcan be processed to determine additional characteristics, such asreflection, color, texture, ambient lighting and the like.

The mapping operation can be performed in any desired manner, before orduring projection output by an application. In one mode, the map isgenerated in response to execution of a setup process for the projector.Alternatively, the map can be generated automatically from registeringits surroundings either before or during projector operation. The mapcan be updated automatically during projector use and/or by updating itwith a new mapping operation at selected or periodic intervals.

The programming is configured for modifying the projection of theoptical output by performing image processing which changes imagecharacteristics encoded within the electrical source signal in responseto the map, or by sending control signals to optical elements (e.g.,focus, zoom, and so forth), to change output based on mappedcharacteristics of the output direction. The programming can performimage processing to modify the size, aspect ratio, focus, color, hue,brightness, and other desired attributes either separately or in variouscombination.

It should be appreciated that information is retrieved from the map inrelation to where the image is being projected, or for a location uponwhich the image will soon be projected. For example, in oneimplementation, the pan and tilt angles are used as indices foraccessing map data. The map can be generated in a mapping processperformed to detect and define characteristics for surfaces within thesurroundings of the image projection element to which output can beprojected upon. One important characteristic to be mapped is distancebetween projector element and nearby surfaces upon which the projectionis to be displayed. It will be appreciated that as a projected image ismoved across irregular surfaces, the distance between the projector andthe surface(s) onto which an image is projected may change. In suchinstances, the focal length may need to be changed as the image is movedacross the surface(s) to allow for proper viewing. For example, considera nearby surface at a distance of x feet from the projector which slopesaway to a surface that is y feet from the projector (where x≠y), inwhich case it likely would be necessary to change the focal length asthe projected output moves along the surface to maintain a focusedimage. Numerous other characteristics of the surfaces can be taken intoconsideration, including reflectivity, texture, color, and ambientlighting. The surface characteristics are looked up from the map, forexample, based on the direction that the projector is pointing, such asby pan and tilt angles. Any convenient means of look-up mechanism can beutilized, such as a table, array, list, and other mechanisms as will beunderstood by one of ordinary skill in the art.

It should be recognized that the projector can be configured to projectany type of image, ranging from still images to full-motion videoimages, or any combination thereof. In addition, the light output fromthe projector can be modulated to form a remote control command sequencefor receipt by devices capable of receiving commands optically (e.g.,visible, near-infrared, and/or infrared optical energy). The opticaloutput from the projector can be generated using any desired technology,such as light sources passing through a variable transparency pixilatedgrid, light sources being directed with micromirror arrays, arrays oflighting elements (e.g., LEDs, OLEDs, and other light generatingdisplays), ray tracing projection units (e.g., laser, LEDs, and soforth), and other techniques and combinations thereof for controllingprojected light output.

By way of example, one projector implementation includes: (a) a lightsource; (b) a modulation element configured for modifying the intensity,color, or a combination of intensity and color, of the light projectedfrom the light source; and (c) an optical assembly for directing thelight projected from the image projection element. It should be notedthat the combination of light source and modulation element areconfigured for converting the electronic source signal into a projectedoptical output.

In one mode of operation, the speed with which the motion controlelement executes the motion is controlled by a speed setting or motionprofile, such as established by the application program or user. Itshould be noted that the speed and motion profile also provide for thegeneration of random speeds, random motions between a “from” and “to”location, ramping speeds, and other variations which can be accommodatedby the programming. By way of example, the pan and tilt speeds can bepreferably set within the range of between approximately 0.25 degreesper second to approximately 60 degrees per second.

While operation of the image projection system is described utilizing acomputer processor for controlling operating aspects, other circuits canbe alternatively employed, such as digital signal processors,multiprocessing arrangements, as well as logic-arrays, custom circuits,and so forth. The processor, or other control device, can be containedwithin a computer device configured for executing the programming inaddition to executing desired application programming. Examples includepersonal computers, workstations, networked computers, gaming computers,multimedia computers, and so forth. Alternatively, all or a portion ofthe computer processing and programming can be incorporated within theprojector device itself, wherein application programming operating on aseparate computer can send operating commands as well as image/videodata to the multi-directional projector.

FIG. 5 illustrates an example of a projection method according to theteachings herein. Mapping is performed as represented in block 100 togather information about surfaces to which the image projector can bedirected for displaying its output. A map is generated and retained inmemory, which is accessible to the system, for these surfaces upon whichan image can be projected. It should be noted that all areas accessibleto projector motion can be subject to projection or, alternatively,characteristics from the map can be used to limit the range of surfacesbased on distance, surface reflectivity, color, ambient light or otherdesirable criterion. For example, it will be appreciated that projectingan image onto select surfaces, such as a slotted air outlet duct oractive halogen light fixture, would be of minimal display value whereinthe system is configured for blocking projection to these areasdepending on parameters set by the user.

To project images, an electrical signal is received and converted to aprojected optical output from the image projector as per block 102.Motion commands are received as in block 104 for controlling motion andcharacteristics of projection. In block 106, the projector output isdirectly manipulated (e.g., focus, zoom, and other optical changes)and/or the image signals are modified (e.g., change size, color, shape,contrast, intensity, and so forth) in response to data looked up fromthe map based on the direction of the optical output being projected.

FIG. 6 illustrates another example of the image projection method. Inthis example, an image is projected from a projection element onto afirst projection location as represented by block 110. As per block 112,a command is received to move the image onto a second projectionlocation. Then, according to block 114, a motion control element coupledto the image projection element is activated to move the output imagefrom the first projection location to a second projection location.Finally, image output is modified as per block 116 in response toaccessing mapped characteristics of the second projection location.

Interfacing with the projection apparatus can be accomplished using anumber of alternative mechanisms. In one embodiment the projectorcontrol software is configured for being executed with API calls madeavailable to one or more source applications operating on the samecomputer. By way of example, the programming can be divided into a SetupAPI and a Control API. The commands in the Setup API are used toestablish (setup) the parameters used during projection control. TheControl API translates commands from the applications to controlprojector operations and also interacts with the image processingroutines to process images accordingly. The use of APIs allows projectorcontrol functions to be rapidly integrated within any desiredapplication programming. By way of example, a Setup API and a ControlAPI are provided in this embodiment.

The following is a partial list of API command information provided byway of example.

Init (Position)

-   -   Description: Resets Projector to position and initializes Image        Processor. Modified Image is black (no image).    -   Parameters: Position: X, Y, Z coordinates (or pan, tilt angles)

Map (File)

-   -   Description: Builds a map of surroundings    -   Parameters:        -   Map File—A description of the space surrounding projector.

Move (From, To, Speed, Display)

-   -   Description: moves projector from the “From” position to the        “To” position according to given speed, or speed characteristic.        In one mode the projected image is black (no image) when        projector begins moving from the “From” position.    -   Parameters:        -   From: X, Y, Z coordinates (or pan, tilt angle values)        -   To: X, Y, Z coordinates (or pan, tilt angle values)        -   Speed: Degrees per second            -   If not defined—generates random, variable speed.        -   Display:            -   True—the projector image stays on during movement. If                the image is moved across an irregular surface and/or                the distance between the projector and the surface                changes during movement, the focal length of the image                projection element may necessarily change to maintain                focus during movement (e.g., maintain focus along an                infinite number of projection locations).            -   False—the projector image goes to black during movement.

Nudge(Horizontal degrees, Vertical degrees, Speed, Display)

-   -   Description: This moves the Projector the number of degrees        specified for both Horizontal and Vertical as the given speed        and with Display on or off.    -   Parameters:        -   Horizontal degrees: 0-360 degrees

Vertical degrees: 0-360 degrees

-   -   -   Speed: Degrees per second            -   If not defined—generates random, variable speed.        -   Display:            -   True—the Projector image stays on during movement. If                the image is moved across an irregular surface and/or                the distance between the projector and the surface                changes during movement, the focal length of the image                projection element may necessarily change to maintain                focus during movement (e.g., maintain focus along an                infinite number of projection locations).            -   False—the Projector image goes to black during movement.

Rotate(Number of times, degrees per rotation, Speed)

-   -   Description: rotates projector in horizontal direction and moves        it up or down with the degrees per rotation.        -   Parameters:            -   Number of times: specifies number of times to rotate.            -   Degrees per rotation: how much to move up/down per                rotation.            -   Speed: Degrees per second.

Stop( )

-   -   Description: stops moving projector, continues to display image.    -   Parameters: None

Continue( )

-   -   Description: Start moving projector with current defined        “movement parameters”.    -   Parameters: None

Random(bounded space, speed, type)

-   -   Description: Move projector randomly in specified movement type.    -   Parameters:        -   Bounded space:            -   If defined, only move the Projector within defined 3D                space.        -   Speed: Degrees per second (whether horizontal or vertical)        -   Type: smooth, jerk, zigzag, and so forth.

Advanced API functions:

-   -   Registering object surface characteristics    -   Video Effects (in, out)    -   Focus (control “in focus” to “out of focus”)    -   Light Intensity (brightness)    -   Fade(black to white, white to black, speed)    -   Zoom (control image size and intensity)

It will be appreciated that the multi-directional projection system andmethod described herein can be implemented according to variations andoperating modes, some of which have been discussed, the following beingprovided by way of further example and not limitation.

In one mode, referred to herein as “time line” mode, the application canplace commands within, or for retrieval by, the projector control modulefor execution based on a time line. The commands may be stored in thesequence portion of memory 66 shown in FIG. 4. For example the time linecan provide global synchronization based on a video time line“HH:MM:SS:FF”. It will be recognized that timing may be performed basedon local timing, or based on common timing signals such as timingsignals received from a Global Positioning System (GPS) satellite.

In one mode of operation of the multi-directional projection system andmethod, the changes in projection direction are generated in response toan audio and/or video input. In this way, the motions of the projectedimages or video are “choreographed” in response to aspects of the audioor video feed. By way of example, a simple “motion audio” mode canutilize averages audio frequency to control vertical projector movementwhile the beat is registered for controlling horizontal movements.However, it should be appreciated that the audio can be analyzed inresponse to registering of any selected characteristics from whichmotion commands are generated. Similarly, the type of image processingperformed can be modulated in response to the audio or videocharacteristics. One of ordinary skill in the art will appreciate thenumerous mechanisms available upon which audio and video can beanalyzed, and thus used for driving motion and image processing withinthe present system. It should be noted that the audio or video input fordriving projection changes can be signals which are directed towardcomponents other than the multi-directional projection system, such asvideo directed at a flat panel display, or audio generated from a musicsystem.

In one mode of operation, the multi-directional projection system isutilized for projecting remote control command sequences and directingthem to any device (e.g., TV, PC, audio system, audio/video player, gameconsole) within the vicinity of the projection system which is capableof receiving optical remote control commands. It should be appreciatedthat the projector can be adapted to perform this remote control aspecteither separately or in combination with other aspects described herein.Remote control commands to be communicated by the projector are encodedwithin all, or a portion, of the image being output by the projector.For example turning all projected pixels between an on-state (e.g., inany portion of the spectrum (color) compatible with the receivingdevice) and an off-state in response to the bit encoding of a commandstring to communicate that command string to all remote control devicesupon which the light of the projected image sequence falls. To providegreater location selectivity, the remote control command can be encodedwithin some portion of the projected output (e.g., a block of pixels)which are directed upon the optical receiver of a remote control device.The location of the optical receiver within the remotely controllabledevice is contained in the map (or a separate map if desired) whereinone mode of the system is adapted to adjust optical or electricalaspects of the image signals based on the map. The projection system isalso configured for adjusting the framing rate of projection output tomatch bit durations of remote control command sequences as required tocommunicate with any specific remote control device receiver. The remotecontrol mode allows application programming to communicate with nearbyoptical remote control devices, thus allowing for example, a gamingapplication to send commands to an audio system, television, or otheroptical remotely controlled device in response to directives from thegaming application. It should be appreciated that the remote controlsignals can be embedded within otherwise conventional images or videosso as to simultaneously provide for both viewing and remote controloperation.

To communicate with nearby remotely controllable devices, theprogramming is configured for: (a) maintaining information on thelocation of one or more devices within the range of the image projectionelement which are responsive to optical remote control commands; (2)maintaining optical sequence information for each optical remote controlcommand to which each of the one or more devices is responsive; and (3)controlling the motion actuation to direct the optical output of theimage projection element toward a given device of the one or moredevices in response to receipt of a remote control command to betransmitted by the projector; (4) generating the electrical sourcesignal by converting the remote control command based on the maintainedoptical sequence information for the given device; and (5) modifyingelectrical source signals based on the map to limit receipt of opticalimage, as necessary, to the given device.

If more than one remotely controlled device is in the vicinity ofprojection, the size of the projected image portion containing thecommand sequence can be reduced so that commands are selectivelycommunicated to a given device. Projection output size can be controlledby using a zoom mechanism (e.g., changing angle subtended by projectioncone), or by reducing the number of pixels from which light isprojected, or a combination of zoom and reducing the number of pixelsfrom which the light is output.

The multi-directional projection system can be configured for operationin any portion of the optical spectrum, such as in the ultra-violet,near-ultra-violet, visible, near-infrared and/or infrared portions ofthe optical spectrum. In the case of communicating with remote controldevices, it will be understood that many conventional optical remotecontrol devices operate in the near-IR, or IR portions of the spectrumwhich may be beyond the spectral range of a visible output projectorelement. It should be noted, however, that controlling non-visiblespectra remote control devices does not require compatible spectraloutput from the entire projector. Communication in these cases can beestablished using as few as one optical element, such as a simple LED(e.g., near-IR, IR, or UV), which can be separate from, or integratedwithin, a visible display array to provide multi-directional output witha spectrum compatible with the optical remote control receiver. Numerousvariations of remote control communication can be implemented by one ofordinary skill in the art without departing from the teachings herein.

Although the description above contains many details, these should notbe construed as limiting the scope of the invention but as merelyproviding illustrations of some of the presently preferred embodimentsof this invention. Therefore, it will be appreciated that the scope ofthe present invention fully encompasses other embodiments which maybecome obvious to those skilled in the art, and that the scope of thepresent invention is accordingly to be limited by nothing other than theappended claims, in which reference to an element in the singular is notintended to mean “one and only one” unless explicitly so stated, butrather “one or more.” All structural and functional equivalents to theelements of the above-described preferred embodiment that are known tothose of ordinary skill in the art are expressly incorporated herein byreference and are intended to be encompassed by the present claims.Moreover, it is not necessary for a device or method to address each andevery problem sought to be solved by the present invention, for it to beencompassed by the present claims. Furthermore, no element, component,or method step in the present disclosure is intended to be dedicated tothe public regardless of whether the element, component, or method stepis explicitly recited in the claims. No claim element herein is to beconstrued under the provisions of 35 U.S.C. 112, sixth paragraph, unlessthe element is expressly recited using the phrase “means for.”

1. An image projection system, comprising: an image projection element;a motion control element, coupled to said image projection element andconfigured for moving an image projected by said image projectionelement from a first projection location to a second projectionlocation; and a control processor configured for: receiving anelectrical source signal and converting the signal to an image forprojection by said image projection element, actuating said motioncontrol element in response to receipt of a motion control command, andmodifying said electrical source signal according to a map of projectionlocation characteristics.
 2. A system as recited in claim 1, whereinsaid control processor is further configured for generating said map. 3.A system as recited in claim 1, wherein said image projection elementcomprises: a light source; a modulation element configured for modifyingintensity, color, or both intensity and color, of light from said lightsource; and an optical assembly configured for projecting said lightfrom said image projection element; wherein said light source and saidmodulation element convert said electronic source signal into said imagefor projection by said optical assembly.
 4. A system as recited in claim1, wherein said motion control element controls horizontal angularmotion, or vertical angular motion, or both horizontal angular motionand vertical angular motion.
 5. A system as recited in claim 1, whereinsaid control processor is further configured for controlling speed ofmovement of said image according to a predetermined speed setting ormotion profile.
 6. A system as recited in claim 1: wherein said mapincludes distance information associated with projection locations.
 7. Asystem as recited in claim 1: wherein said map includes informationassociated with optical characteristics of projection locations; andwherein said optical characteristics are selected from the group ofoptical characteristics consisting of distance, reflectivity, texture,color, and ambient lighting.
 8. A system as recited in claim 1, furthercomprising: a zoom actuator for modulating projection cone angle outputby said image projection element; wherein said control processor isconfigured for controlling said zoom actuator in response to receipt ofa zoom control command, or in response to mapped characteristics ofprojection locations, or in response to both receipt of a zoom controlcommand and characteristics of projection locations.
 9. A system asrecited in claim 1, further comprising: a focus actuator for modulatingfocus of said image projection element; wherein said control processoris configured for controlling said focus actuator in response toinformation retrieved from said map for a given direction of imageprojection.
 10. A system as recited in claim 1, further comprising: asensor for registering projection location characteristics to which saidimage projection element can be directed.
 11. A system as recited inclaim 10, wherein said sensor comprises an optical imaging device or arange finding device, or both an optical imaging device and a rangefinding device.
 12. A system as recited in claim 1, wherein modifyingsaid electrical source signal comprises processing signalcharacteristics associated with image size, aspect ratio, focus, color,hue, or brightness, and any combination thereof.
 13. A system as recitedin claim 1, wherein said control processor comprises a computer deviceconfigured for executing projection control programming and applicationprogramming.
 14. A system as recited in claim 1, wherein said controlprocessor is further configured for: maintaining information associatedwith location of one or more remote devices within the range of saidimage projection element which are responsive to optical remote controlcommands; maintaining optical sequence information associated withoptical remote control commands to which said remote devices areresponsive; controlling said motion control element in response toreceipt of a remote control command to be transmitted by said projectionsystem to a said remote device; and modifying said electrical sourcesignals based on said remote control command and said map to limitreceipt of a projected image to a given device.
 15. A system as recitedin claim 1, wherein said image projection element is adapted forprojecting an image in the visible, near-infrared and/or infraredportions of the optical spectrum.
 16. An image projection system,comprising: an image projection element; a motion control elementcoupled to said image projection element for controlling horizontalangular motion, or vertical angular motion, or both horizontal angularmotion and vertical angular motion of an image projected by said imageprojection element; a focusing lens assembly coupled to said imageprojection element; an optical detector element coupled to said imageprojection element; said optical detector element configured forregistering distance to a projection location to which said imageprojection element can be directed by said motion control element; and acontrol processor configured for: generating a map of distances toprojection locations in response to information received from saidoptical detector as a function of pan angle and tilt angle, receiving anelectrical source signal and converting the signal to an image forprojection by said image projection element, actuating said motioncontrol element to move an image projected by said image projectionelement from a first location to a second location in response to thereceipt of a motion control command, and modifying focal length of saidfocusing lens assembly in response to distance information in said mapassociated with projection locations.
 17. A system as recited in claim16: wherein said control processor is further configured for processingsaid electrical source signal to modify image size, aspect ratio, color,hue, brightness, or any combination thereof, in response to a map ofprojection location characteristics; and wherein said electrical sourcesignal is modified by said control processor prior to projection of animage by said image projection element.
 18. An image projection method,comprising: projecting an image onto a first projection location;receiving a command to project the image onto a second projectionlocation; moving the image from the first projection location to thesecond projection location; and modifying the image in accordance withmapped characteristics associated with the second projection location.19. A method as recited in claim 18, wherein modifying the imagecomprises changing optical characteristics of projection, or modifyingsignals carrying the image prior to projecting the image onto the secondprojection location, or both changing the optical characteristics ofprojection and modifying the signals carrying the image.
 20. A method asrecited in claim 18, wherein modifying the image comprises changing afocal length in response to retrieving a distance value from the mappedcharacteristics.
 21. A method as recited in claim 18, further comprisingmapping the characteristics associated with the second projectionlocation.
 22. A method as recited in claim 18, further comprising:moving the image from the second projection location to the firstprojection location; and modifying the image in accordance with mappedcharacteristics associated with the first projection location.
 23. Amethod as recited in claim 18, further comprising: moving the image fromthe second projection location to a third projection location; andmodifying the image in accordance with mapped characteristics associatedwith the third projection location.
 24. A method as recited in claim 23,wherein the image is directed across a projection surface from the firstprojection location to the third projection location.
 25. A method asrecited in claim 18, wherein the image comprises a moving image.